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ABSTRACT 

Outcomes of doctoral education at a P r iy«*te t urban 
university were studied^ and a questionnaire was developed^ A total 
of 707 Ph.D. and Ed.D. recipients from 15 departments who had 
graduated Between 1963 and 1984 were surveyed to determine^ the 
impact of doctoral education on career development, the perceived 
influence of the institutional values on graduates, and the frequency 
and type of research or scholarly activities engaged in during and 
after graduate school. Data from the 168-item survey were analyzed by 
department, degree, and four general academic fields, and a number of 
indices were developed • The career development index included the 
following variables: relationship of degree to job and career, and 
postgraduate career path. Variables included in the institutional 
values index were: concern for ethical values, interest in religious 
beliefs, and influence of the university on personal values . 
Activities to indicate scholarship included: submitting grant 
proposals and articles for publication, receiving external funding 
for research, and delivering papers. The analysis includes 
comparisons for graduates in the f ields of education , humanities, 
social sciences, and medical sciences. (SW) 
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An Institutional Study 



Abstract Shis study sought to establish base-line data on the outcomes of 
doctoral education at a private , urban umversity. Through the use of a survey 
instrument developed by the researchers, Hi.D. and Ed.D. recipients from 16 
departments who had graduated between 1963 arid 1984 were contacted. Three 
primary areas were examined: 1) the impact of doctoral education on career 
development, 2) the perceived influence of the institutional values on 
graduates, and 3) the frequency and type of research/scholarly activities 
engaged in during and after graduate school. Data provided by 707 respondents 
to the 168 item survey were analyzed by department, by degree and by four 
general academic fields, and a number of indices were developed. The main 
thrust of this paper is the development of the instrument and the indices. 
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Attitudes about the benefits of a college education are changing. At the 
undergraduate level, a "retxirn^n-investinent 11 standard has become the Treasure 
of the quality of a baccalaureate degree. How iirpbrtant is that cr iteri on at 
the graduate level? Ihe answers are inportant not only to the individual 
graduate but to aditiinistrators in higher education as well. For one private, 
urban university, the need to identify the perceived outcomes of doctoral study 
precipitated a year long investigation of 16 doctoral degree granting 
departinents and their graduates. 



While much research has been done on the outcomes of undergraduate 
education, the same attention has not been given to graduate education, with 
this concern in mind, the dean of the graduate school and the dean of the 
sdhbbi of education sponsored a study to determine the outlines of doctoral 
education at a private urban imiveraity in the Midwest, classified as a 
Research University 11 by the earnegie classif ication. 

The meas ur e ment of OTtodmes is by nature an imprecise endeavor, even if we 
taiow exactly which dimensions to assess (Kinnicdc, 1985; ifeoe, 1985) . Because 
the main "product" of universities is the educated person, there are 
innumerable problems in the evaluation process that other organizations do not 
face. Distinguishing between inputs and outputs is difficult. Problems arise 
in comparing individual graduates in one university and across institutions and 
in simply identifying desired outcomes. Since the majority of outcome studies 
have concentrated on undergraduates, there are few reliable models to use. 
Because of these inherent problems in evaluating outcomes of i^versities, 
theory has an important place in trying to tsrderstand cnitcomes in academic 
organization. Development of a conceptual framework for studying outcomes of 
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academic organizations, hcwever, also poses difficulties. 

There is no single right way of measuring educational outcomes because 
dif ferent kinds of outcomes are the results of different educational 
experiences and intentions. Measurement involves careful consideration of the 
match between assessment and setting (Ewell, 1985) . In general, there are two 
distinct levels of congruence between setting and assessment design that are 
integral to the study of outcomes. First, the form and content of assessment 
should be consistent with the institution's distinctive mission and educational 
objectives. Second, the tools and techniques that are used in assessment must 
correspond to what the student has actually experienced. 

In addition, the demand for specificity affects the units of analysis to be 
used. For studying the outcomes of graduate education, the appropriate unit of 
analysis is the college or the department, not the institution as a whole. The 
department level is where experiences actually occur and the unit with which 
students actually identify. The college level is the administrative level 
which most directly affects departmental objectives. 

Purpose 

Sis study sought to identify and urxierstand the relationships between the 
self-reported values and activities of graduates of doctoral progr ams and three 
broadly-defined outcomes of gr aduat e school. 

The three outcome areas examined in this study were: 

1) The impacc of doctoral education on the career development patterns of 
Ph.D. and Ed.D. graduates. 

2) The perceived influence of the value orientation of the institution on 
these graduates. 

3) The frequency and type of researciy scholarly activities engaged in 
during and after graduate school by Hi.D. and Ed.D. recipients. 
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Conoeptnal Framework 
Much of the conceptual framework of the study Was derived frara rontemporary 
enpirical research on college effects at the urdergraduate level {Astin, 1984; 
Feldman and Newcanib, 1969; Tinto, 1975; Weidman, 1984). Three general 
constructs were used to develop the outcome assessment. The first construct, 
career development, was defined as the fit between the individuals 1 intentions 
or expectations for graduate study and their subsequent career patterns. The 
second construct, institutional value orientation, focused on indicators of the 
iiifluence of fee institution's liberal arts orientation and tradition of moral 
and humanistic education on the mdividuais 1 graduate school experiences. The 
last construct was scholarship, the spirit of either adding to the body bf 
existing knowledge in an academic field or disseminating new knowledge, which 
is the central outcome of the research-oriented doctoral program. Indicators 
of this construct were type and frequency of various scholarly activities of 
the graduates during graduate study and currently. This construct was further 
categorized using Gouldner f s cosmopolitan-local typology. Those activities 
that increased the knowledge base axv3/or that were more traditional forms of 
dissemination were labeled as cosmopolitan. 



After a search of the literature on the caitcames of a college education 
resulted in a preponderance of undergraduate studies and a few single-program 
studies, several months were spent in the development of a survey instrument. 
The instrument has fcxir basic sections \tfiidh deal with the follcadng topics: 
career develapn^t, research and sdhbiarly activity, values and life style, and 
demographic information. Questions were used from completed studies of 
undergraduate career patterns and those concerned with the effect of 
iiistitutibnal values on student develop ment . The questions centering bh 
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sciioiariy or research activities during and after graduate school were 
developed after soliciting in p ut fram the chairs of the 16 departments with 
doctoral p r og ra ms. 

The first section dealing with career develqprnent cxantains 23 questions. 
This section concentrated on identif icatidh of di ara c ter isti^ specifically 
related to the graduates 1 current eonplqyment positions and an determination of 
perceived relationships between career characteristics and graduate studies. 
Thus, questions about type of employment position, setting of employment, arid 
current gross annual salary were asked. Questions about perceived influence on 
career, quality of graduate preparation for career responsibilities , impact of 
graduate studies on mobility and advancement, and reason for obtaining a 
doctoral degree elicited the information to complete the career development 
component. 

Section two deals with scholarly and research productivity arxi contains 65 
questions vSidh are arranged in three main groups. The first group requests 
information on activities engaged in during doctoral studies. Since the 
professional activities of faculty mentors is seen as an important factor in 
professional socialization, i±te second group of questions deals with 
respondents 1 perceptions of faculty advisor activities and atti t udes toward 
research. The final group of questions is focused can present activities of the 
respondents. The construction of this section relied heavily on input from 
various departments, since different disciplines value different activities, 
for example, publishing books versus articles, receiving research grants, 
serving on study groups for government agencies, and so forth. 

The third section included a variety of topics ranging from general 
satisfaction with life to the perceptions of the graduates about the impact of 
specific institutional values on their am values and subsequent behavior. The 
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institution's undergraduate cu±ricular enpiasis is the liberal arts* Questions 
investigated the influence of this curricular mission oh the doctoral 
education. For example, the respondents were asked to determine the impact of 
each of the specific humanistic goals on their doctoral experiences, using a 
Likert scale of positive to negative. The inpact of the institution's 
tradition of moral education was the secssnd institutional value measured. 
Questions in this set included degree of concern for ethical issues and 
justice, and the inpact of the university's orientation on personal goals, 
Biere were 43 items in this section. 

She final section of 37 questions was designed to collect denegraphic 
information about graduates' currant status and the conditions under *ftiidh they 
pursued their studies. There was also a place for respondents to make general 
comments concerning their doctoral education. 

The survey went through five revisions based on responses from c ur r ent 
graduate students, doctoral degree holders and experts in questionnaire 
construction. Two forms were used in order to check on the reliability and 
validity of the instrument. The order of the questions varied on the two 
forms. Form A was distributed to 75% of the graduates, and Form B was 
distributed to the other 25% of the population, A t-test performed on the Form 
B responses and a sample of the Form A responses showed no significant 
difference in the responses on the two forms, A test of equivalence on the two 
forms resulted in a coefficient of .71. Analysis and follow-tp interviews of a 
stratified sample of nbn-reqpOTdCTts suggest that the data are representative, 
reliable, and valid. 

in February 1985, the 15 page q ues tionnaire was sent to 1050 alumni who had 
received doctoral d e gr e es in 16 departments between Janua ry 1963 and January 
1984 . The questionnaires were pre-coded so that a f oiiw-up mailing oouid be 
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dbneti The follcw-up mailing three wee3cs later resulted in 707 l^esponses, and 
ah additional 28 surveys v/ere returned because of bad addresses; Seventy 
percent of the 1022 questiorffiaires were returned, with the departmental 
response rates ranging from 46% to 80%. 

Data 

A data set of 168 variables for the 707 respondents contains a wealth of 
information. The typical respondent in this study is a white (91%) male (67%) 
between the ages of 41 to 50 (39%) who is currently married (73%) . During 
graduate school, he went full-time (57%), received financial assistance (68%), 
and took five years to complete his doctoral studies (25%) . 

Ihe 16 departments were categorized into one of the four academic fields 
in the following manner (number's indicate total number of resporxients) : social 
sciences (172) included psychology and sociology; humanities (108) included 
classics, English, history, and piilbsqphy; sciences (117) included anatorty, 
biochemistry/physics , chemistry, mcfobioiogy, piarmadblogy, and physiology; 
education (298) included administration, foundations, curriculum, elementary, 
counseling psychology and higher education. 

Indices 

Three general indices were developed to measure (1) the impact of the 
degree on career development, (2) the impact of the institutional values on 
graduates, and (3) ths scholarly orientation of graduates to their fields. The 
basic process of constructing the indices consisted of six steps. Firsts a 
question was formulated which expressed the dimension the index would measure. 
Then the research team used face validity to choose variables which related to 
each of these questions. The responses to these questions were then re-coded, 
usually using a positive (+1) , neutral (bj , negative (-1) framework. 

Next a coefficient of reliability (axshbadh f s alpha) was ccatputed. This 
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coefficloit is a measure of the interoorrelation of the variables. The goal of 
the index cx>nstrurtibn was to dbtaih a maximum &< with the fewest variables, 
arid variables were manipulated until this goal was reached. Once the variables 
in the index were determined, the sura of their values was computed. 



The career development index is a measure of the co ntribut ion doctoral 
education has made to the accomplishment of career goals. It consists of nine 
Variables, including: the relationship of degree to job, the impact of the 
degree on career, the post-graduate career path, graduate preparation for 
career responsibilities , recxammendation of the program to others, and the 
contribution of the doctorate to salary, advancement, job security, and career 
mobility. The reliability coefficient for this index was .7687 and the index 
has a possible range of -8 to 9. For the 707 responses in this study, the 
range was -7 to 9 with a mean of 4.55 and a standard deviation of 3.16. 

The second index was constructed to measure the influence of the 
institutional value orientation. The items selected here relate graduates 1 
perceptions of the impact of institutional values on their own values. Twelve 
variables were identified as appropriate to this construct. The 12 items are: 
concern for ethical values; interest in religious beliefs; influence of the 
university oh personal values; the influence of each of these university goals 
on doctoral experience: be a Christian influence in the world, be a partner to 
the city, develop a passion for justice, cxsrtfront major problems of the day, 
foster a spirit of inquiry, serve the neighborhood and aanraunity, emphasize the 
development of the whole person, value gathering and disseminating knowledge, 
and uphold the institutions tradition of moral and humanistic education. The 
reliability coefficient was .8792 and the range for this index was a score from 
-26 to 29. For the 707 respondents in this stud/, the range was -13 to «?9, 
with a mean of 9.64 and a standard deviation of 7.76. 
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The third and final construct was developed using indicators of sc^larship 
reported by the graduates of these doctoral programs. After reviewing the 
literature on productivity and surveying the 16 department chairs and graduate 
program advisors, 18 activities were identified as representative of scholarly 
behavior. In constructing an outcome index for scholarship, however, the 
behaviors were categorized as either those activities of the sore traditional 
notion of adding to the body of knowledge in a field or these activities of 
disse minatin g knowledge to the field. She drem^blitan scholarship index 
included 10 activities, namely: sttoitting articles for publication, having 
articles published, having articles published in referred jcxmrals, submitting 
grant proposals, receiving ©eternal funding for research, delivering papers, 
being an invited participant in a syirposium, serving on a goverimint s t udy 
group, participating in a research grant, and serving on an editorial board. 
The reliability coefficient for this index was .8891. The possible scores in 
this index rancid from 0 to 48. For the group of respondents as a whole, the 
mean was 11.78 and the standard deviation was 11.26. 

For the local scholarship index, eight activities were included: 
consulting in the field, teaching seminars, receiving professional awards, 
attending conferences, holding office in a professional organization, 
organizing a symposium, participating in a research project, and reading books 
in the field. The reliability coefficient for local scholarship was .7793, and 
the index has a range of 0 to 24* The resulting mean for the group was 10.83 
with a standard deviation of 5.11. 
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Results 

This study has allowed the researchers to examine a wide variety of issues. 
Table 1 gives responses from a sainple of departments to the question of vfriether 
graduates would recommend their doctoral program to others. 

Tfcble 1 

Recommend Graduate Program to Others 



Department 


Yes 

Enthusiastically 


Yes 


Probably 
Not 


NO 


ft 


17.6% 


64.7% 


17.6% 




B 


25.0 


50.0 


20.8 


4.2% 


e 


18.5 


55.6 


25.9 




D 


20.6 


50.0 


14.7 


14.7 


E 


15.8 


57.9 


15.8 


10.5 


F 


56.3 


43.8 







Hiis item alone does not constitute a program evaluation, but it does highlight 
areas to be examined. 



In reviewing doctoral programs, it is important to understand student 
motivation in pursuing advanced study. Table 2 summarizes reasons for pursuing 
the doctorate reported by academic field. The majority of graduates of the 
social sciences and education shared "cradentialing 11 as their primary reason 
for pursuing a doctoral education while humanities and medical sciences 
graduates gave an academic or research interest as their reason. 
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Table 2 

Reason for Pursuing Doctorate 



Med Sciences 
n=117 



Education 
flF=298 



Humanities 
n=108 



Soc sciences 
n=L72 



Credential ing 



61% 



27 



23% 



58 



67% 



24 



Research Interests 
Commitment to the 

Insti t ut ion 6 8 6 

Note. Col umns do not total 100% due to missing data and 



29% 
58 
§ 

errors. 



Analysis by degree allowed comparison of Ph.D. and Ed.D. recipients within 
the field of education. While there are some differences in the pre-graduate 
school employment and part-time, full-time patterns of graduate school 
attendance, the two groups shew remarkable similarities in almost all ether 
aspects. Ed.D. graduates are somewhat more concentrated in school (grade and 
high) settings and, as Table 3 shows, are twice as likely to work as 
administrators. 



Table 3 

Current Employment Position by Degree 



Ph.D. 
n=145 



Ed.D. 
n=148 



Teacher 



Ed *"ttti ■' n \ wt-.rafcra* 

Researcher 



Management 
Professional staff 
Self-employed 



28.3% 
23.4 
4.8 
6.9 
27.6 
4.1 



18.9% 
49.3 

6.8 
19.6 
2.7 



Note. Columns do hot total 100% due to missing data and rounding errors. 
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As part of the set of ij^catbrs of scholarship^ respondents were asked to 
indicate the amount of faculty emphasis an sc^iarship and/or research they 
perceived in their departments viiile in graduate school. A strong faculty 
emphasis was perceived by 70% of the medical sciences respondents (Table 4) . In 
education only 32% reported a strong emphasis an researcjy scholarship by- 
faculty and 25% perceived minimal to no emphasis during their studies. 

Table 4 



Emphasis on Researciyscholarship by Faculty of Graduate Department 





Education 
rP298 


Humanities 


Soc sciences 
n=172 


Med Sciences 
n=117 


Strong emphasis 


32% 


45% 


52% 


70% 


Same emphasis 


40 


36 


39 


22 


Minimal emphasis 


18 


10 


8 


4 


No emphasis 


7 


3 


2 


3 



Tables 5 through 8 report the results of one-way analysis of variance 
using the three construct indices and academic fields. Tables 5 and 6 show 
that there were significant differences among Jie four academic fields for both 



i n di c es of scholarship. Medical science graduates have the highest average for 
current cosmopolitan scholarship and graduates in social science have the 
highest average on the local sdiolarship index. 
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Table 5 
Cosmopolitan Scholarship 



Academic Field n Mean SD df F ratio 

Education 298 10.02 9785 3 10.673* 

Humanities 108 11.06 9. 72 

Medical Sciences 117 16.76 13.37 

Social Sciences 172 11.90 11.88 

*p<.001 was chosen because of the relatively large sample size. 



Table 6 
Local Scholarship 

Academic Field n Mian SD df F ratio 

Education 298 10.81 5732 3 8.781* 

Humanities 108 9.06 4.61 

Medical Sciences 117 10.64 4.78 

Social Sciences 172 12.20 4.91 

*p<.001 was dhosen because of the relatively large sample size. 



A one-way AN07A was done using the career development index and academic 
field. Table 7 shows that social scientists reported the highest positive 
impact averages on career development, and those in the humanities reported the 
lowest. There is a correlation between these results and the reasons given for 
pursuing the degree (Table 2) • 
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Table 7 

Difference in career Development by Academic Field 



Academic Field n Mean SD df F ratio 

Education 298 4736 3?22 3 13.205* 

Humanities 168 3.55 3.45 

Medical Sciences 117 4.22 3.18 

Social Sciences 172 5.74 2.45 

*p<.001 was chosen because of the relatively large sample size. 



The index for the institutional value orientation was analyzed by academic 
field. Those in the medical sciences reported a significantly lower score than 
those in the other three fields (Table 8) . 

Tfeble 8 

Difference in Institutional Value Orientation by Academic Field 



Academic Field 


n 


Mean 


SD 


df 


F ratio 


Education 


298 


9.79 


8.13 


3 


8.672* 


Humanities 


108 


11.29 


7.97 






Medical Sciences 


117 


6.7Q 


7.21 






Social Sciences 


172 


10.74 


6.7Q 







*p<.001 was chosen because of the relatively large sample size. 
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Discussion 

The effect of the Eh.D. on career patterns was generally positive for all 
respondents. Social science respondents perceived the greatest positive 
influence of the degree oh their careers. The largest nuniber of graduates in 
this academic field came from psychology and they identified CTedentialing as 
their primary reason for pursuing a doctorate. Bieir siibsequerxt career Voices 
as professional staff in hospitals and universities and as private c onsulta nts 
help clarify the view of the Eh.D. as a positive influence. Those in education 
also perceived the doctorate as a career advancement credential. Over a third 
of the respondents in this academic field are currently administrators in K-12 
school settings where the terminal degree has not until recently been an entry 
level requirement. This may account for the positive perceptions of the 
degree's influence on career development. The humanities have suffered in 
recent years with an bversupply of Hi.b.s, so it is not surprising that the 
respondents in this field, vAio are pr^amirantly academicians, were the least 
positive about the iinpact 6f their Eh.D. ah the develcpiKait of their careers. 
Most of the respondents in the medical sciences are currently \rorking in two 
areas, teaching at the university level or doing research in private industry. 
In both settings, having a doctorate is a valuable asset in career advancement. 

The iinpact of the specific institutional mission on the values or 
behaviors of doctoral recipients seems slight, within the choice process, 
prospective graduate students lock for indicators of academic departmental 
quality and reputation, not for general institutional values. While university 
administrators might hope for a greater iinpact of institutional values, the 
index itself is not an absolute measure. What does became clear is that the 
medical sciences graduates perceived little influence of the institutional 
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values while the other three fields were Sore positive about the ef f set of 
these values. The m edical sciences campus is removed from the other campuses, 
arid there is almost a total absence of contact with the other seg men t s of the 
university. This may partially explain the differences in this index. 

The last outcome studied, scholarship, is specific to the doctoral 
degree. In examining the carandtment of graduates to scholarly values and 
activities, several issues are relevant. Most important is the effect of the 
academic discipline upon the type of scholarly behavior the graduates observed 
in their doctoral programs and the importance of scholarship within their 
academic field. The medical sciences had the highest score on csDsncpolitan 
sc^larship while the social sciences scored highest in local scholarship, it 
is difficult to determine if the piiiosqEihy of this tmiversity, car the 
expectations of the disciplines, or the orientations of these specific 
departments are responsible for the differences in scholarly productivity. An 
analysis of variance was done on the departments within each academic field for 
both indices of scholarship. Education was the only field in which a 
significant difference existed. The departments within the field of education 
showed significant differences both in the cosmopolitan and the local indices 
of scholarship. 

Conclusions 

As a result of this study, the institution has a much clearer picture of 
its doctoral graduates. A follow-up study will be conducted in five years in 
order to compare results and depict trends. In addition, new doctoral students 
in the same 16 departments completed questionnaires in the fall of 1985 
designed to elicit ajprppriate entry level data. This process will be followed 
each subsequent fail. When the information obtained at the point of student 
entry into a program is combined with the results of the five year follow-up 
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study, the insti t ut ion will have a better tmderstaiSing of the impact it has 
onabctoral st udents . This is the first phase of the graduate schools effort 
to measure outcomes. 

It would also be helpful to be able to compare results from this study 
with outcomes research done at other universities. It is only through the use 
of a wide variety of comparative data, both longitudinal and across 
institutions, that meaningful outcomes information will be accumulated. 

Little work has been done to assess the outcomes of doctoral education. 
Two primary contributions of this study are the design aid testing of an 
instrument to evaluate outcomes at tills level and the development of indices to 
measure the iinpact of the doctoral program on career development, values and 
socialization to the academic field. She data collected has given the 
ins t i t uti on new insists into the s tren gt hs and limitations of its doctoral 
p r ogr ams . There are many other ways in which this data can be analyzed, and 
the process of analysis and interpretation continues. While the study deals 
only with one institution, it makes a contribution to instrument development 
and measurement of the three constructs involved. 

In addition to studies that focus, on program design and curricular and 
research requirements, outcomes studies have a significant contribution to make 
to educational evaluation and planning. Any attempt to redesign or refocus 
graduate programs should be informed by knowledge of the iinpact of the previous 
arxi current practices of the institution. 
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